1. Technical Field
Embodiments of the invention generally relate to Peripheral Component Interconnect Express (PCIE) buses, and more particularly to a method and apparatus for providing cable redundancy and failover for multi-lane PCIE IO interconnections.
2. Related Art
The Peripheral Component Interconnect (PCI) standard was first introduced in the early 1990s. By using a PCI bridge chip connected to a frontside bus and a processor, PCI provides direct access to a system memory within a computer system for any peripheral devices connected to a PCI bus. The PCI bridge chip regulates the speed of the PCI bus independent of the speed of the processor such that a high degree of reliability can be achieved.
The PCI Express (PCIE) standard is the successor to the PCI standard, the pertinent of which is incorporated herein by reference. Compared to PCI, PCI Express can achieve a higher transmission rate with fewer physical pins. Unlike the previous generation PCI busses, the PCI Express uses a point-to-point bus architecture. Accordingly, a dedicated bus is used for data transaction between any two devices that use a PCIE bus system. The dedicated bus is facilitated by a switch which establishes the point-to-point connection between the communicating devices. Thus, the switch is used as an intermediary device and is physically and logically located between any two devices attached to the computer system.
The PCIE switch includes a plurality of ports to facilitate the attachment of the devices to the computer system. A physical connection between a device and a port of the switch is commonly referred to as a link. Each link includes one or more lanes, and each lane is capable of transmitting data in both directions. Hence, each lane is a full-duplex connection.
A link which includes a single lane is called an x1 link. Likewise, a link that includes two lanes or four lanes is called an x2 link or an x4 link, respectively. PCI Express allows for interfaces with different widths, such as x1, x2, x4, x8, x12, x16 and x32 links, to meet the different bandwidth requirements of various peripheral devices. Thus, a dedicated bus may be 1-lane, 2-lane, 4-lane, 8-lane, 12-lane, 16-lane or 32-lane wide.
Contemporary server class computers frequently use PCIE IO adapters as a primary IO adapter technology. Often CPU enclosures include a limited number of PCIE adapter slots to customize the IO options of a particular server. However, the CPU chassis packaging typically limits this to very few such slots while compute power per chassis has grown substantially with multi-core processor chips. Commonly, servers provide mechanisms to connect a CPU to PCI adapter slots in one or more additional “IO Expansion” chassis. For example, in PCIE systems, a PCI Root Port (PRP), also referred to as a PCIE Host Bridge (PHB), is a component of the CPU electronics and creates a PCI bus that connects to either a single PCIE IO adapter slot directly or to a PCIE switch in an IO expansion chassis that expands that PHB bus to multiple PCIE adapter slots below that switch. IO expansion requires placement of one or more PCIE adapter slots in the expansion chassis and connect those slots to PHBs within the CPU chassis.
Typically, since these IO expansion chassis are physically different electronic chassis or enclosures, the electronic connections between the PHBs within the CPU chassis and PCIE adapter slots within the IO expansion chassis requires physical cabling between these enclosures. These cables may require more than several inches of interconnect distance between a PHB and a PCIE adapter slot and may even extend between different physical racks containing CPU and IO expansion enclosures.